Evaluating Evaluation: an Empirical Examination of Novel and Conventional Usability Evaluation Methods

Software designers and software users have different views of software. Designers see all the parts that make up a system, and which are usually hidden from the users, who only see the interface. Conventional usability evaluation methods strive for objectivity in their measurement of user/system interaction, yet it is often the subjective experience of using a system which is important. Whiteside et al (1988) propose contextual research as a way of getting hold of this type of data, but a number of problems present themselves. It is difficult to observe users in context without intruding and altering the nature of the interaction, especially as much of the interesting parts of an interaction are internal to the user, and not available for direct observation. Methods such as 'thinking aloud' verbal protocols (which may make such phenomena explicit) are both intrusive, and likely to alter the phenomena being observed.